Sufficient Conditions for Uniform Stability of Regularization Algorithms

In this paper, we study the stability and generalization properties of penalized empirical-risk minimization algorithms. We propose a set of properties of the penalty term that is sufficient to ensure uniform β-stability: we show that if the penalty function satisfies a suitable convexity property, then the induced regularization algorithm is uniformly β-stable. In particular, our results imply that regularization algorithms with penalty functions which are strongly convex on bounded domains are β-stable. In view of the results in [3], uniform stability implies generalization, and moreover, consistency results can be easily obtained. We apply our results to show that `p regularization for 1 < p ≤ 2 and elastic-net regularization are uniformly β-stable, and therefore generalize.